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Summary 

Two versions of a prototype Free Flight 
cockpit situational display (Basic and 
Enhanced) were examined in a simulation at 
the NASA Ames Research Center. Both 
displays presented a display of traffic out to a 
range of 120 NM, and an alert when the 
automation detected a substantial danger of 
losing separation with another aircraft. The 
task for the crews was to detect and resolve 
threats to separation posed by intruder aircraft. 
An Enhanced version of the display was also 
examined. It incorporated two additional 
conflict alerting levels and tools to aid in 
trajectory prediction and path planning. Ten 
crews from a major airline participated in the 
study. Performance analyses and pilot 
debriefings showed that the Enhanced display 
was preferred, and that minimal separation 
between the intruder and the ownship was 
larger with the Enhanced display. In addition, 
the additional information on the Enhanced 
display did not lead crews to engage in more 
maneuvering. Instead an opposite trend was 
indicated. Finally, crews using the Enhanced 
display responded more proactively, tending 
to resolve alerts earlier. 

Introduction 

NASA Ames Research Center is investigating 
displays for cockpit situational awareness, with 
the hope of assisting the implementation of 
Free Flight. This work is in response to the 
RTCA Task Force 3 Report on Free Flight 
Implementation 1 , which identified cockpit 
situational awareness displays as a key 
component of the next generation air traffic 
management (ATM) system. This report states 
that “The architecture and technology on 
which the emerging ATM system is based 
makes increasingly heavy use of new displays 
that provide flight crews with real time 
situational awareness.” Therefore, the RTCA 
report recommends the immediate initiation of 
“the development of standards for a cockpit 
situational awareness display of traffic 
information.” Determining these standards 


requires a detailed human factors evaluation, 
where the nature and format of the displayed 
information are examined. 

While the exact functions supported by a 
cockpit situational display (CSD) depend upon 
the operational concept that underlies Free 
Flight, a degree of crew responsibility for self- 
separation resides in almost all proposed Free 
Flight concepts (see the RTCA Task Force 3 
Report' for a discussion of Free Flight 
evolution). Early in the development of Free 
Flight, conflict calculations, conflict 
resolutions, and flight replanning will 
probably be done on the ground by the ATC, 
with alerts and instructions issued to aircraft by 
the ATC (in cooperation with the airline 
operation center, AOC). 

However, the role of the flight crew will be 
significantly enhanced, even in this early 
system. Their most likely role, which the CSD 
must support, will be: 1) to monitor for safe 
and effective flight: and 2) to make requests of 
the ATC consistent with these criteria. In the 
safety case the CSD may allow the crew to view 
potential losses of separation. Thus, in active 
coordination with the ATC, the pilots could 
use the CSD as an aid in detecting and 
resolving potentially dangerous situations. In 
the effectiveness case the CSD may allow the 
crew to optimize other criteria by requesting 
potentially advantageous re-routings. Later in 
the development of Free Flight, 
telecommunications and cockpit 
automation/display advances may make it 
possible to give more discretionary authority 
to the cockpit crew. In this case the role of 
the ATC would be 1) to provide safe operating 
constraints upon flight paths; and 2) to 
monitor safe flight and intervene if a 
dangerous situation occurs. In between these 
two points, and within different flight regimes 
(e.g., enroute and terminal area) differing 
mixes of authority and responsibility may be 
imagined. 



Almost any envisaged CSD must meet certain 
design goals. First, the CSD should provide the 
crew with sufficient predictive information so 
they can respond to potential conflicts in a 
proactive, and not reactive, manner. That is, a 
CSD should give a crew the information and 
tools that allow them to monitor and evaluate 
developing conflict situations, and to select 
candidate maneuvers well before such 
decisions become critical. Therefore, the 
display should incorporate a system that gives 
good advance information about possible 
conflicts (e.g., an alerting system) and also 
provides the crew with means for resolving 
conflicts (e.g., a path planning tool). 

However, while providing a display that 
incorporates good advanced conflict detection 
may be an important, it is not a sufficient, 
criterion for a good CSD. The crew of an 
aircraft may know about goals and concerns 
that are unavailable to the automation, and 
therefore be sensitive to aspects of the evolving 
traffic pattern which escape the automation. 

For example, crews may get information in 
flight briefings that is unavailable to 
automation, or get updated information from 
voice communications (e.g., party line 
information about weather and other 
problems). Such information may be about 
changes in intended paths of other aircraft 
(revised flight plan), or about factors that can 
be reliably expected to influence these flight 
plans (e.g., reports of turbulence at some 
altitude). In addition, crews learn from 
experience. In order to evaluate the impact of 
these factors the crews need the ability to 
project the present situation into the future, 
and not just rely upon the automation to 
inform them of potential problems (conflict 
predictions) and the set of possible resolutions 
(resolution advisories). How far into the future, 
and what subset of traffic the crew must be 
able to monitor/extrapolate is a subject for 
debate and investigation. 

Finally, to effectively provide the crew with 
this information, display design must be 
highly concerned with format issues. Simply 


placing the information on a display is not 
enough. The display must not be cluttered, it 
must be easy to monitor and remember, and it 
must never lead to confusion. In addition to 
these objectives, this must be accomplished in 
a manner that keeps workload low (the crew 
must not be heads-down and “flying the 
display”). 

The work reported in this paper focused on 
the development and examination of an initial 
CSD prototype for commercial enroute 
airspace operations along with its preliminary 
evaluation in a high fidelity Boeing 747-400 
flight simulator. This work took place within 
the context of a larger Free Flight 
Demonstration Simulation. The goals and 
results of that larger effort will not be reported 
here. Instead, this report will focus on the 
design, and initial evaluation, of the prototype 
CSD. 

Design Goals 

The first part of this report documents the 
function, structure, and rationale for our 
prototype CSD. The design effort focused on 
meeting three design goals. The first design 
goal was to provide a basic display that 
captured the fundamental elements of the 
present traffic situation, i.e., traffic location, 
altitude, heading, climb/descent, and identity, 
while attempting to manage clutter. Direct 
display of traffic speed was not used in this 
first prototype since the use of speed almost 
always involves extrapolation, and 
extrapolation by humans is quite error prone 2 . 
Therefore, speed information was handled by 
prediction/extrapolation tools made available 
to the crews. 

The second design goal was to develop an 
alerting system that could provide crews with 
advanced information about potential 
conflicts. This would allow crews to begin 
planning for the resolution earlier, to resolve 
conflicts earlier, or both. This design goal is 
controversial however. On the one hand, early 
predictions could lead to unnecessary 
maneuvering as crews respond earlier in an 
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attempt to avoid low probability potential 
encounters. On the other hand, early 
maneuvering can be more efficient than late 
maneuvering with the crew being able to select 
more gradual maneuvers when given more 
time and distance to the potential conflict. 
Furthermore, early predictions can remove 
both the surprise and heightened workload 
that accompany alerts to imminent conflicts. 

The third design goal was to provide tools that 
allowed the crews to more effectively predict 
the evolution of the present traffic pattern, and 
to predict the consequences of potential 
maneuvers that they might make. Thus we 
were attempting to provide tools to give the 
crew 4-D situational awareness based upon 
both the present situation and upon potential 
situations. 

Many design issues were not addressed in this 
early design prototype. For example, we did 
not attempt to include tools that might directly 
provide crews with resolution advisories based 
on measures of fuel or time efficiency. 

Instead, we looked at their abilities to resolve 
conflicts by determining their own maneuvers. 
Also, since problems posed by Free Flight 
concepts are most tractable in the enroute 
sectors (due to the congestion in the terminal 
area), we have initially concerned ourselves 
with designing a display that would be useful 
in the enroute environment. For this reason 
this initial prototype is called the Airborne 
Management of Enroute Separation (AMES) 
display. 


Evaluation Goals 

The second part of this report describes the 
simulation evaluation of the prototype CSD. 
This simulation was designed to surface issues 
and provide future research direction on CSD 
development and evaluation - it was not 
designed to allow definitive conclusions about 
the overall adequacy of the prototype CSD, 
nor about the efficacy of Free Flight using 
these displays. 

The simulation evaluation focused on two 
related goals. The first goal was to determine if 
our general approach to CSD design was on 
the right track. Our approach to this was to 
compare two versions of the CSD within a 
highly idealized flight environment, using 
objective data from a set of candidate 
dependent measures, and subjective data taken 
from crew debriefings. One version of the 
CSD contained a minimum of functionality 
(e.g., traffic depiction and a simple conflict 
alert) while the second contained additional 
enhancements that met the design goals 
described in the previous section. This 
simulation was designed to examine the global 
display design, and not to not gather detailed 
performance data about the value of the 
individual elements within the CSD. 

The second goal of the simulation was to 
examine the sensitivity of some candidate 
performance measures. Selection of 
dependent measures in realistic or quasi- 
realistic flight simulations is difficult if the 
goal is to provide a measure of safety. For 
purposes of maintaining legal separation. 
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Figure 1. Control panel. 
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safety has been defined as maintaining 
specified separation criteria. On the other 
hand, a more general definition of safety 
(or its inverse, threat) is almost certainly a 
much more multidimensional concept, 
especially in a Free Flight environment. 

For example, the RTCA has expanded the 
area of concern to a much larger (although 
as yet undefined) alert zone based on to- 
be-determined criteria 1 . Furthermore, the 
sensitivity of legal separation as a 
dependent measure is very low in scenarios 
that do not immediately place an aircraft 
into a critical situation. Therefore, since 
this safety concept has not yet been 
defined, and since the legal measure of 
safety is of low sensitivity, candidate 
dependent measures were selected on the 
basis of their potential to illuminate the 
relative impact of the CSD on certain types 
of performance (e.g., degree of proactive 
response) and not on safety per se. These 
dependent measures will be described and 
discussed in the Methods section. 

AMES Traffic Display Description 

The AMES display is composed of three 
fundamental parts. First there is the 
symbology associated with the basic traffic 
display. Second there is the symbology 
associated with the conflict alerting logic. 

Third there is the symbology associated with 
the display tools that help the crew examine 
potential future traffic scenarios. 

Basic Traffic Display 

The basic traffic display was built upon 
functionalities present in the Map Mode of the 
Visual Navigation Display present in the 
Boeing 747-400. This display is shown in 
Figure 1 in an expanded forward view with a 
160 mile range, but was also available to the 
crews in a full (360°) compass mode, and with 
display ranges going from 20 NM to 640 NM. 
However, although the cockpit display could 
be set to longer ranges, the effective simulated 
range of the display was limited to the 120 
NM range that the RTCA has suggested for the 
proposed Automated Dependent Surveillance 
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Figure 2. Basic traffic display. 

Broadcast system (ADS-B) systeml. The 
display depicts the traffic as elongated open 
white triangles showing aircraft headings and 
associated data tags. All data tags contain 
relative altitudes (fix 100), and also contain up 
and down arrows (T4-) that indicate when an 
aircraft is climbing or descending. The data 
tags may also contain aircraft ID if a pilot 
wishes. The pilot could toggle the IDs for all 
aircraft on and off using a button ( 1DENT) on 
an additional control panel (see Figure 2). 

Due to programming constraints, it was not 
possible to toggle on and off individual IDs in 
this simulation, although this was known to be 
a potentially valuable display decluttering 
option. 


1 The ADS-B system is, at present, the most well 
developed candidate for an airborne surveillance system. 
The RTCA ADS-B MASPS (Draft 5.0) presently requires a 
minimum surveillance range of 90 NM with a desired 
range of 120 NM for cooperative separation in oceanic 
and low density enroute airspace. 


4 


i 



5 


None 

(SAO) 


te 

UJ 


4 


3 


< 

cn 


o 

E- 


2 


1 



Low 

(SA1) 


0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 

Probability of Conflict if No Maneuver is T aken 


0.9 


Figure 3. Threat level as a function of conflict probability and number of available effective 
maneuvers. 


Conflict Alerting 

An evolving threat concept guided our design 
of the conflict alerting displays. This followed 
from our desire to provide the crew with as 
much forewarning as possible about a 
potential threat. To accomplish this, we took 
advantage of the fact that we can heighten 
attention without necessarily sounding an 
alarm. When an aircraft exceeds some 
minimal threat potential it can be highlighted. 
This has several advantages. First, at greater 
distances much smaller maneuvers can lead to 
effective avoidance. Second, at greater 
distances the crew has time to monitor and 
plan for eventual maneuvers if they become 
necessary. This removes time pressure and the 
need for an emergency response by the crew. 
By providing multiple stages the crew can 
monitor threat evolution and decide when and 
if it is necessary to take action. 

The threat status of all aircraft within the 120 
NM ADS-B surveillance range are mapped 
into six threat levels - four Free Flight 
situational awareness (SA) levels, and two 
TCAS (TC) levels. These threat levels are 
computed by a Free Flight conflict algorithm 
developed by Yang and Kuchar at MIT 3 and 
by the TCAS II algorithm. The Free Flight 
threat levels are related to potential, or 
occurring, ‘alert zone’ contacts. The alert 


zone concept, originally proposed in the 
RTCA Task Force 3 Report 1 , defined an alert 
zone in terms of a temporal and/or spatial 
safety envelope surrounding an aircraft. Thus, 
when an aircraft penetrated the alert zone of 
another aircraft, this ‘alert zone contact’ 
signified a significant potential for loss of 
some minimal separation between those 
aircraft. However, where the original RTCA 
alert zone concept is based on temporal and/or 
spatial envelopes around an aircraft, the alert 
zone concept used in Yang’s and Kuchar’ s 
algorithm was based on two different threat 
components, although these are still strongly 
affected by temporal and spatial proximity. 

The two threat components used by Yang and 
Kuchar are shown in Figure 3. The threat 
component defined as “probability of conflict 
if no maneuver is taken” corresponded to the 
probability of a simultaneous loss of 5 mile 
horizontal separation and 1000 ft vertical 
separation between aircraft if no avoidance 
maneuver is taken. Estimates of conflict 
probability are taken from the output of an 
airborne conflict alerting logic produced by 
Yang and Kuchar. This logic produces 
estimates of the probability of conflict (loss of 
separation) between one’s own and other 
aircraft (traffic). This logic uses probabilistic 
extrapolations of the present trajectory of 
Ownship and traffic to generate a probabilistic 
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distribution of future Ownship and traffic 
locations. For Ownship the random 
component modeled in the extrapolation is 
assumed to come solely from random physical 
disturbances, primarily wind fluctuations. The 
distribution of this random component is 
assumed to be normal and to generate track- 
aligned fluctuation in speed (a = 15 kt), and 
fluctuation in cross-track error (0=1 NM). 
These specific parameters were estimated from 
data obtained empirically from traffic by 
Paielli and Erzberger 2 * 4 . Extrapolations of 
traffic and Intruder positions assumed these 
same random processes, but also included 
another two additional components to capture 
deliberate (intent driven) changes in altitude 
and heading. In these components the 
probabilities of a tum or an altitude change 
were modeled as Poisson processes, with mean 
rates of 4 tums/hour and 4 altitude 
changes/hour. When they occurred, the 
magnitudes of these changes were randomly 
drawn from distributions in which 1) heading 
changes between 5° and 20° were equally 
likely, and heading changes less than 5° were 
less likely; and 2) altitude changes between 0 
and 10,000 ft were equally likely. 

The threat component defined as Total 
Available Maneuvers corresponds to the 
number of maneuvers (climb/descend, tum 
left, tum right, speed up, slow down) that will 
bring the conflict probability below 0.05. In 
this study 5 maneuvers were defined together 
with maximum maneuver sizes. Two of these 
maneuvers were standard rate turns (left or 
right), with a maximum 30 degree tum being 
allowed. Another two maneuvers 
corresponded to increasing or decreasing true 
airspeed by up to 50 knots2. A climb or 
descent at lOOOft/min counted as the fifth 
maneuver. Climbs and descents were 
combined into a single maneuver category 
since conflicts could always be resolved if 
either or these were initiated at least one 


2 No simulated winds were present so true airspeed was 

equal to ground speed. Also, speed increases of up to 50 

knots were allowed, but not always possible for a 
particular initial speed and flight level. 


minute ahead of time. Thus resolutions using 
climbs and descents remained available much 
longer than did resolutions involving speed or 
heading maneuvers. Thus it was decided, in 
effect, to decrease the weighting on the altitude 
maneuver (since it did not reflect maneuvering 
constraints until very late) by collapsing both 
climbs and descents into a single maneuver 
category. 

All maneuvers was assumed to be preceded by 
probabilistic delay with a mean of 1 minute 
and a 95% probability of being less than 2 
minutes, and with the maneuver being 
instantaneously effective when it did occur. 
This was done to model the crew flight 
response latency to a conflict, and thus give 
the crew sufficient time to coordinate with 
other aircraft and/or the ATC. 

No vehicle dynamics were incorporated in the 
conflict alerting software, and instantaneous 
changes in heading, altitude rate, and speed 
(after a delay of 1 min) were assumed. 
However, this delay plus immediate change 
was probably a close approximation for turns 
and altitude changes since most turns and 
altitude rate changes could be made within 10 
to 20 seconds. On the other hand, this cannot 
be said for the speed changes since it takes 
fairly longer intervals to change speed by 50 
knots (and sometimes it cannot be done at all). 

Prior to, or at the time of an alert zone contact, 
the crew was expected to evaluate and resolve 
it by deciding among three actions. These 
actions are 1) no action, 2) maneuvering or 
negotiating a maneuver with the other aircraft, 
or 3) by handing off the problem to the ATC 
who will then direct the required maneuver. 
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Table 1. T1 

ireat Level 

Description, A] 

ert Status, and Symbology. 

THREAT 

ID 

SAE 

Alert 

Category 

ALERT 

STATUS 

THREAT 

LEVEL 

SYMBOLOGY 






SA„ 

0 

No Alert 

No Threat 

White Open Triangle 

SA, 

0 

No Alert 

Low Threat 

Blue Open Triangle 

SA, 

0 

No Alert 

Mod Threat 

Blue Filled Triangle 

sa 3 

1 

Alert Zone 
Contact 

High Threat 

Blue Filled Triangle, Solid Ownship 
Route line. Solid Intruder route 
line, and solid outlined Airspace 
boundaries, Blue ALERT message 
with estimated time to closest 
approach (CTA) 

TC, 

2 

TCAS 

Traffic 

Advisory 

Very High 
Threat 

Amber filled Diamond, Blue 
ALERT message with estimated 
time to closest approach (CTA) 

TC 2 

3 

TCAS 

Resolution 

Advisory 

Very High 
Threat 

Red filled Diamond, Blue ALERT 
message with estimated time to 
closest approach (CTA) 


The final two threat levels are taken from the 
TCAS II algorithm (Traffic Alert and 
Resolution Advisory). This algorithm 
generates four threat levels, but only the two 
most serious levels were used in the present 
implementation. The threat levels 
corresponding to the two least serious TCAS II 
advisories were not used in the display, since 
the functions of these SA levels were 
effectively provided by the Free Flight SA 
levels. However, the two most serious TCAS II 
alerts were included since they signaled 
progressions beyond an SA 3 , and they 
provided resolution information that was not 
included in the Free Flight SA levels. 

Each Free Flight and TCAS threat level had its 
own unique associated display symbology (see 
Table 1 and Figures 1, 4-8). The symbologies 
for the Free Flight levels were selected so that 
they would not conflict (be similar in color or 
shape) with the symbologies already in use 
within the 747 navigation display. The 
symbologies for the final two levels were the 
same as those used for the two highest TCAS 
II alert levels. 


Table 1 also shows how these threat levels, and 
their associated symbologies and aural alerts, 
can be mapped directly into the alert 
categories specified in the SAE recommended 
practice for flight deck alerting systems 5 
shown in Table 2. The first three Free Flight 
Situational Awareness (SA) threat levels ( SAo, 
SA h SA 2 ) represent no threat to moderate 
threat and are intended to be simply 
informative. As the threat of any aircraft on 
the display increases across these levels its 
associated symbolic depiction changes as 
indicated in Figures 1, 4, and 5 and Table 1. 

The next threat level, SA 3 , is associated with the 
onset of an alert zone contact (Figure 6). An 
SA 3 level is assumed to reflect a significant 
threat to which the crew must devote their 
attention and to which they may have to 
respond. Consistent with this need to bring an 
SA 3 to the crew’s attention, when an alert zone 
contact first occurs only the aircraft positions 
and altitudes remained on the display with the 
alert symbology. In some cases this required 
the removal of symbology associated with the 
display tools (described below) as well as the 
removal of all non-intruder aircraft IDs. The 


7 



Table 2. SAE Alerting Categories. 


SAE 

LEVEL 

CONDITION 

CRITERIA 

ALERTING SYSTEM 
CHARACTERISTICS 


AURAL 

VISUAL 

3 

Emergency 

Conditions which require 
immediate corrective or 
compensatory action by 
the crew 

Required 

Red 

2 

Abnormal 

Conditions which require 
immediate crew 
awareness and 
subsequent corrective or 
compensatory crew 
action 

Required 

Amber 

1 

Advisory 

Conditions which require 
crew awareness and may 
require crew action 

Permitted 

Optional (not red) 

0 

Information 

Conditions which require 
flight deck indication, 
but not necessarily as 
part of an alerting system 

None 

Green, blue or white 


SA 3 symbology was composed of the SA 2 
symbology plus collision route lines extending 
from Ownship and the aircraft posing the 
threat. These route lines terminated with 2.5 
NM radii circles at the estimated point of 
closest approach. Circles with this radius were 
used since a visual overlap to two such circles 
would indicate a violation of the 5 NM 
protected zone surrounding each aircraft. The 
word “ALERT”, was presented in large blue 
letters at the bottom right of the display along 
with the estimated time (in minutes and 
seconds) to closest approach (CTA), and 
remained on until the threat levels dropped 
below SA 3 . An audible advisory (“Alert Zone 
Transgression”) was presented simultaneously 
with the display change. If, at this time, the 
crew wished to return any of the eliminated 
symbology for the non-involved traffic, they 
could do so. 

The final two threat levels (TC h TC 2 ) are 
associated with the two highest TCAS threat 
levels and use the standard TCAS symbologies 
and audible warnings. In addition the ALERT 
message remains on. 


Display Tools 
Predictor Tool 

Pilots face a significant problem when called 
upon to extrapolate a present traffic situation 
into the future since there is often considerable 
variability in the human prediction of motion 2 . 
Therefore, without the type of training and 
experience given to air traffic controllers, most 
pilots will require some form of automated 
aiding. Although the alerting logic relies 
upon computational extrapolations to assess 
threat, it only makes this assessment available 
to the crew in terms of the assigned threat 
levels. Thus, this logic alone does not provide 
the crew with a picture of how the traffic 
situation will evolve over time. Instead it 
leaves the crew completely dependent on the 
basic display plus alerting logic for both threat 
evaluation (in detection and resolution) and 
for future situational awareness. In other 
words, this logic leaves a gap between the 
detection of an aircraft that may pose a 
problem at some specific time and place in the 
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Figure 4. Situational awareness level 1 (SA,). Blue is 
depicted as bold white. 



Figure 5. Situational awareness level 2 (SA 2 ). Blue is 
depicted as bold white. 


future (which the logic provides), and the 
understanding of the evolving traffic pattern, 
including other non-alerted aircraft, that is 
needed for resolutions (which the logic does 
not provide). The predictor tool is designed to 
fill in this gap by allowing a pilot to 
extrapolate the positions of all aircraft on the 
display. A pilot accomplishes this by first 
selecting the predictor tool using the 
“PREDIC” button on the control panel 
(Figure 2). Then a pilot may choose to 
display predicted route lines by setting the 
temporal predictor to a value between 0 and 
10 minutes (see dotted line in Figure 7). This 
prediction is based simply on the assumption 
that the aircraft will continue along their 
present track, at their present speed, for the 
specified period of time. No turn rate 
information is included or assumed in this 
display. The predicted route lines are depicted 
as solid lines with the same color as the 
associated aircraft symbol. These lines project 
from the front of Ownship and from the front 
of all other aircraft. They extend out distances 
corresponding to the specified number of 
minutes. Centered on the end of each route 
line is a 2.5 NM radius circle depicting the 
projected location of the protected airspace 
region at the end of the prediction interval 
(Figure 7). If crews ensure no overlap in 


protected regions then a 5 NM separation is 
assured. 

Route Assessment Tool 

In a system where the crew may be responsible 
for selecting new routes, the crew needs to be 
able to evaluate the safety of proposed course 
modifications. The Route Assessment Tool 
(RAT) helps the crew to do this by providing 
the means to generate and test potential 
changes to vertical speed, path speed, and 
heading. The RAT is enabled using the 
CONFLICT selection button on the control 
panel (Figure 2) and allows a pilot to probe 
for clear paths. While in the RAT Mode a pilot 
may examine various heading (± 30 deg), 
speed (± 50 knots) and altitude rate (± 1000 
ft/m) changes using the dials on the control 
panel. 

When the RAT is enabled a white dashed 
“what-if’ route line is displayed in addition 
to the present route line, and the word “RAT” 
appears in large white letters at the bottom 
right of the display together with indications 
of the present values of the RAT speed and 
vertical speed settings (Figures 8 and 9). The 
“what-if’ route line extends out for a 
projected 10 minutes (the same as that used 
for “what-if’ conflict detection while in the 
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Figure 6. Using the route assessment tool (RAT) and 
showing a new SA 2 conflict along a potential path. 
Blue is depicted as bold white. 


RAT Mode) and, in combination with the 
conflict detection logic, is used to assess 
potential conflicts along this proposed path. 
When various SA Levels are triggered by the 
RAT the same symbology used for standard 
conflict alerting is employed with the 
following exceptions. For a RAT induced SA 3 
(Alert zone contact) no audible alert is 
sounded, no ALERT message is displayed 
(unless there is already an ALERT in 
progress), and the display will not be cleared 
of any aircraft IDs or of any RAT or Predictor 
Tool symbology. 

If a true SA 3 alert occurs while a pilot is using 
the RAT, the RAT is disenabled and a 
decluttered alert depiction is displayed (see 
below). 

The RAT was designed to assume that 
Ownship maneuvers would become 
instantaneously effective 1 minute in the 
future. This is instead of the probabilistic 
function with a mean of 1 minute used for the 
standard alerting logic described earlier. For 
example, when at time t, a pilot examined the 
consequences of a 10 degree turn, the RAT 
would approximate this maneuver as an 
instantaneous 10 degree tum at time t+1 . 
Furthermore, due to limitations in the RAT 
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Figure 7. Situational awareness level 3 (SA 3 or Alert 
Zone Contact). Blue is depicted as bold white. 

software, only RAT advisories from this time 
(t±l) forward were displayed. Thus the RAT 
did not generate SA levels and advisories for 
the initial first minute of the prediction. 

Simulation Evaluation of AMES 
Display 

A simulation examining these display concepts 
during en route flight was conducted using the 
Boeing 747-400 Level D flight simulator 
located at the NASA Ames Research Center 6 . 

The purpose of the simulation was to provide 
initial data on the ability of crews to conduct 
en route Free Flight and resolve conflicts from 
the flight deck. In most ways the simulation 
reflected optimistic assumptions about the 
quality of surveillance information, alerting 
systems, and the ease of air-to-air 
communications. The difficulty of the flight 
scenarios was also minimized in the 
simulation. Specifically, the simulation 
incorporated the following features: 

1. Error free surveillance information (traffic 
positions, velocities, headings, vertical 
speeds, and altitudes). 

2. No weather, no winds. 

3. No extraneous heading or speed 
maneuvering by traffic aircraft (traffic 
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Figure 8. Situational awareness level 1 with predictor 
tool set at 6 minutes. Blue is depicted as bold white. 


aircraft, with the exception of specific 
Intruder aircraft used to create threats, 
never changed direction, or speed). 

4. Traffic density was moderate to low (never 
more than 10 aircraft within surveillance 
range). 

5. No linkage between SA levels and 
probability of a conflict with Intruder. 
Thus there were no false alarms. So when 
any SA or alert level occurred, the 
involved aircraft was always on a collision 
course. 

6. Crews had access to immediate cooperative 
radio communication with any aircraft 
they could see on the Cockpit Situational 
Display. 

7. No secondary tasks (very low workload). 

8. Crews could expect at least one conflict 
per scenario. 

9. All aircraft were assumed to have similar 
equipage, although no Intruder aircraft 
ever used this equipage until contacted by 
the Ownship crew. 

10. All aircraft shared a single frequency but 
there was only a small amount of 
extraneous “chatter” unrelated to 
potential conflicts with Ownship. 

The effect of many of the above features was 

to assume highest quality surveillance and 

alerting information, extremely reliable and 
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Figure 9. Using the RAT to find a clear path 
resolution to an SA 3 alert. Blue is depicted as bold 
white. 

easy flight deck to flight deck 
communications, very little dynamic traffic 
complexity, a low surprise factor to conflicts, 
and no distractions. However, there were also 
some aspects that may have made the task 
more difficult than might be expected in flight 
operations. Among these are the following: 

1. No cardinal altitude rules were assumed, 
thus making head-on conflicts possible. 

2. There was minimal experience with the 
system (about 2 hours of training). 

All flight scenarios presented crews with one 
of two cockpit situational displays. These two 
displays, a Basic and Enhanced version 
described in the next section, relied upon a 
simulated ADS-B surveillance system. This 
simulated ADS-B system broadcasts 
information to, and receives information from, 
all aircraft within a 120 NM surveillance range 
twice a second. The information being 
broadcast contained altitude, x-y location, 
heading, ground speed, and aircraft ID. The 
crews were given an initial briefing in which 
they were told that all flights were either from 
JFK to SFO, or the SFO to JFK, and that all the 
scenarios took place in the Denver Center 
airspace between the Rock Springs VOR 
(OCS) and the Scott’s Bluff VOR (BFF). At 
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the beginning of each scenario they were 
given a specific initial position and flight path. 
During the scenario the crews were told that 
they could expect their CSD to alert them to a 
potential conflict, and that they would then 
have to resolve it safely. They were also 
informed that failure to resolve the conflict in 
a sufficiently timely manner could result in an 
ATC intervention canceling Free Flight and 
returning control to the air traffic controller. 
They were told that this intervention would 
only occur after a SA 3 alert (‘Alert zone 
contact’) and only if they were not 
satisfactorily handling the conflict. Crews 
were told to assume that they could contact 
any aircraft that they could see on the CSD by 
simply addressing that aircraft on the radio, 
and that all aircraft would be tuned to the same 
frequency. Crews were told that all aircraft 
were equipped in a manner similar to 
themselves (i.e., with the same type of CSD 
and ADS-B transmitting and surveillance 
system). 

Dependent Measures 

Since the crews were instructed to stay within 
required separation standards (5 NM lateral 
and 1000 ft vertical), dichotomous 
classification into success and failure at that 
task is an obvious measure of interest. 
Unfortunately, as mentioned in the 
Introduction, this binary measure is relatively 
insensitive for most simulation studies, and 
indeed, minimal separation was never lost in 
any of the scenarios. 

However, other dependent measures, 
specifically, minimum separation, response 
proactivity, and number of Ownship 
maneuvers, are still of interest. These were 
selected since 1) separation is a critical 
concept, 2) the display was designed to 
provide advanced alerts that allow proactive 
responding, and, 3) the RAT allowed 
evaluations of whether maneuvers would result 
in a clear path that required no further 
maneuvering. However, none of these 
variables is a simple measure of the quality of 
overall performance, so caution must be 


exercised in generalizing from these analyses 
(e.g., creating too much separation may lead 
to inefficient flight; responding too early may 
generate unneeded maneuvers; more and 
smaller maneuvers may be more adaptive in 
cases of higher uncertainty). 

Minimum separation . Minimum co-altitude 
separation between Ownship and the Intruder, 
and between Ownship and the rest of the 
traffic (Context Traffic) were examined. Two 
other possible measures of separation, slant 
range separation, and vertical separation when 
aircraft were laterally within 5 NM of each 
other, were not used. Slant range was not used 
since vertical and lateral proximity are so 
unequally weighted when evaluating effective 
separation. The vertical separation measure 
was not used, since aircraft only infrequently 
came within 5 NM laterally of each other. 

Therefore the determination of minimum 
separation between the Intruder and Ownship 
was accomplished by calculating lateral 
separation at all moments that the two aircraft 
were not vertically separated, and then 
selecting the minimum of these data points. 
Instead of the instructed 1000 ft vertical 
separation minimum, 900 ft was used in the 
analysis to allow a suitable margin of error. 

Minimum separation between Context Traffic 
and Ownship was determined using a two step 
process. First the minimum separation was 
calculated for each of the Context Traffic 
aircraft using the same process described 
earlier. Then the smallest of these minimums 
was used as the measure of the minimum 
separation of the Context Traffic from the 
Ownship. 

Response Proactivitv . This measure assessed 
the degree of proactive control by crews. To 
do this the maximum conflict probabilities 
between Ownship and the Intruder were 
examined. These conflict probabilities were 
the estimated probabilities of conflict if no 
maneuver was taken and were supplied by the 
alerting algorithm (see the previous section on 
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Conflict Alerting). These probabilities are 
relatively proportional to the time remaining 
before a predicted conflict. As such, this 
measure reflects how close in time a conflict 
was allowed to approach before it began to be 
resolved (either by either the Intruder or the 
Ownship). In all of the analyses Response 
Proactivity will be plotted as the value of 1.0 
minus the maximum conflict probability since 
this number more closely reflects proactivity 
(a larger number is more proactive). 

It is very important to keep in mind that this 
measure is not a good absolute indicant of 
level of threat or danger. It is only one 
dimension of the two dimensions included in 
the overall conflict logic, excluding the 
dimension reflecting the amount of 
maneuvering that remains available. 

Number of Ownship maneuvers . The mean 
number of discrete commanded changes to 
Ownship heading, altitude rate, and speed were 
examined in separate analyses. Commands 
were categorized as single discrete commands 
when separated by at least 20 seconds during 
which time no new commanded changes in 
these vehicle states occurred. For example, if 
commanded heading started to change, 
remained stable for 10 seconds, changed 
again, and then remained stable for 30 
seconds, this would be considered a single 
heading command. Examining number of 
Ownship maneuvers is meaningful, since it 
indicates if the increased information and tools 
available in the Enhanced display led to more 
or less maneuvering. 

Method 

Design 

The study employed a 2 (Display Mode) by 8 
(Scenario) mixed factorial design. Display 
Mode was the between subjects factor with five 
Boeing 747-400 line crews from a major 
airline in each of the two between subjects 
groups. Scenario was the within subjects 
factor with all ten crews presented with the 
eight pre-programmed conflicts. 


Scenario Construction 
In each scenario, Ownship had an initial 
altitude of 33000 feet, and an initial ground 
speed of 473 knots. During each scenario the 
flight plan segment of Ownship included no 
turns, no altitude changes, and no speed 
changes. Each of the eight scenarios was 
constructed by randomly placing Context 
Traffic within several intersecting flight 
corridors that were located, in turn, within the 
relevant part of the Denver Center airspace. 
Each corridor was 25 to 30 miles wide and was 
located in the 29000 ft to 37000 ft range. 
Traffic could be traveling in either direction 
along the corridor, and had 1000 ft vertical 
separation (e.g. 29000, 30000, .. 37000). 

Thus these scenarios reflected no cardinal 
altitude rules and reduced vertical separation 
minimums. Ground speeds ranging between 
400 and 500 knots were randomly assigned to 
these aircraft, and 1-2 Context Aircraft 
changed altitude per scenario. All aircraft 
were assumed to be of similar type and 
equipage. A sufficient number of Context 
Traffic were generated so that between 6 and 
10 of the Context Traffic were always in view 
on the traffic display during the course of 
each scenario. Scenarios varied in duration, 
with the shortest scenario being 12 minutes 
and the longest being 22 minutes. Table 3 
lists the eight conflicts used in the study. 

These conflicts were generated by the 
deliberate placement of Intruders. The 
descriptions in Table 3 show the sequence of 
SA levels through SA 3 if Ownship does not 
maneuver. Table 3 also shows which aircraft is 
presumed to have the right-of-way. The right- 
of-way “rules of the road” employed were 
taken from FAA FAR Part 91. These rules 
are: 

1. In any head-on encounter both aircraft 
should maneuver to the right. 

2 . In any co-altitude non-head-on encounter 
the aircraft on the right has the right-of- 
way. 

3 . In any co-altitude overtake encounter the 
aircraft in the lead has the right-of-way. 
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designated Intruder was cleared. 


The FARs state no specific right-of-way rules 
for encounters during altitude changes, so no 
such rules were assumed or stated to the crews 
during their briefings. If the Ownship did not 
maneuver, the Context Traffic never generated 
a conflict with Ownship, Intruder, or each 
other. 

Task 

Crews were required to fly eight enroute 
scenario segments. They were informed that 
during these scenarios there would be no 
cardinal altitude rules, the vertical separation 
minimum would be 1000 feet (reduced from 
the standard 2000 ft high altitude minimum), 
and the lateral separation minimum was the 
standard 5 NM. To accomplish this they were 
allowed to maneuver freely and did not have 
to seek the ATC’s approval, although they 
were free to communicate with the ATC at any 
time. The role of the ATC and of the crews in 
the other aircraft was played by a single 
experimental confederate who was a former air 
traffic controller. Crews were also informed 
that all aircraft were assumed to share a single 
radio frequency and that they could 
communicate with any other aircraft by simply 
addressing it using its aircraft ID. They were 
also told not to assume that the ATC or any 
other aircraft could overhear such 
communications (i.e., if they wanted the ATC 
or any other aircraft to know something then 
they had to address the ATC or that aircraft 
directly). When experimental crews did 
communicate with other aircraft the 
experimental confederate would respond. 
Scenarios ended after the conflict with the 


Procedure 

An initial briefing was held for each crew in 
which the experimenters described, 1) the task; 
2) the FAA VFR “rules of the road” for 
conflict resolution; and 3) the display tools (if 
any). Following this, each crew received 1-2 
hours of initial training in which they became 
familiar with the display and the display tools. 
The crews then flew the eight experimental 
scenarios, which took between 2 and 3 hours. 
Following this the crews were debriefed. 

Results 

Global Analysis : The minimum separation, 
response proactivity, and maneuvering data for 
Intruders were examined using a 2 x 8 
(Display Mode by Scenario) mixed analysis of 
variance (ANOVA). The same analysis was 
conducted on the minimum separation data 
for the Context Traffic. The two levels of 
Display Mode were 1) Basic (SA 3 and TCAS II 
SA levels, and no tools) and 2) Enhanced (SA, 
- SA 3 and TCAS II SA levels, predictor tool, 
and RAT). The eight levels of Scenario 
correspond to those listed in Table 3. 

Targeted Analysis : A more targeted ANOVA 
was used to examine these same dependent 
variables in Scenarios 1-4, but only for the 
Intruder and Maneuver data. These scenarios 
are of special interest, because they reflect a 
factorial combination of three factors: Display 
Mode, Direction, and Range. Here, Direction 
refers to whether an Intruder approaches from 
the left or the right, and Range refers to 
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Figure 10. Mean minimum separation with Intruder versus Display Mode and Scenario. 
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whether the Intruder was on a conflicting path 
at a Strategic or Tactical range. Therefore 
these data were analyzed using a 2 x 2 x 2 
mixed ANOVA. The particular value of this 
analysis is that it allows for a more direct 
evaluation of the roles of Display Mode and 
Range. 

Performance data from one crew for one 
scenario (Scenario 2, Basic Display Mode) 
were accidentally not recorded, and for 
statistical analyses these data were replaced 
with data averaged over the other four crews in 
the Basic Display condition for Scenario 2. 

Intruder Data 

Minimum Separation: Figure 10 shows the 
mean minimum separations between Ownship 
and Intruder. The global ANOVA showed 
statistically significant main effects for 
Scenario (F(7,56)=2 1.074, p < 0.001) and 
Display Mode (F( 1,8)= 19.422, p = 0.002), 
and a significant Scenario by Display Mode 
interaction (F(7,56)=2.907, p = 0.012). 
Examination of the interaction shows that 
crews with the Enhanced display had larger 
minimal separations in the strategic lateral and 
head-on scenarios than did crews with the 
Basic display. The mean size of these 
separations looks large if one thinks of the 
separations as coming solely from co-altitude 
passing maneuvers. However, these 
separations often resulted from Ownship or the 


Intruder changing altitude. When this 
occurred early, it could lead to large minimum 
separations (e.g., climbing 1000 feet when 50 
miles from the Intruder yields a minimum 
separation of 50 miles). 

The targeted analysis yielded no significant 
effects, although the effect of Range was 
marginally significant (F(l,8)=4.213, p = 
0.074). 

Response Proactivitv : Figure 11 shows 
response proactivity as a function of Scenario 
and Display Mode. The global analysis 
showed statistically significant main effects for 
Scenario (F(7,56)=2 1.074, p < 0.001) and 
Display Mode (F(l,8) = 19.422, p = 0.002), 
and a significant Display Mode by Scenario 
interaction (F(7,56) = 2.907, p = 0.012). 
Examination of the interaction shows that 
crews with the Enhanced display had more . 
proactive (earlier) resolutions in the strategic 
lateral and the overtake scenarios than did 
crews with the Basic display. 

The targeted analysis yielded significant main 
effects for Display Mode (F( 1,8)= 16.649, p = 
0.004) and Range (F(l,8)=139. 176, p < 

0.001), and a marginally significant main 
effect for laterality (F(l,8)=5.055, p = 0.055). 
There was also a significant Display Mode by 
Range interaction (F(l,8)= 10.563, p = 0.012). 
These effects showed that resolutions were 
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Figure 11. Response Proactivity to Intruder versus Display Mode and Scenario. 
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more proactive when the crews had an 
Enhanced Display, when the conflicts were 
Strategic, and perhaps when the conflicts 
occurred on the right. The interaction showed 
that the effect of the Enhanced display was 
larger when the conflict was Strategic. That is, 
the Enhanced display led to a greater increase 
in response proactivity when there were 
Strategic conflicts than when there were 
Tactical conflicts. 

Discussion : The global analysis showed that 
when Ownship had an Enhanced display the 
resolutions in the overtake and the two 
strategic lateral scenarios were more proactive. 
This is not surprising since the temporal 
differences between the SA, and the SA 3 were 
greatest in these scenarios. That is, there was 
much more opportunity for proactive 
responding in these scenarios. 

On the other hand, the two tactical and two 
altitude scenarios allowed little time between 
these two events (In the altitude scenarios 
Ownship never needed to maneuver. Instead, 
the Intruder was scripted to always maneuver 
at a specific location to avoid a conflict). 

As would be expected, the targeted analysis 
also showed more proactive responding with 
Enhanced displays and with Strategic conflicts. 
Additionally, it showed that the size of this 
Display Mode effect depended substantially 
on whether or not there was a long enough 
range (i.e. r a Strategic range). It is not 
surprising that the advance alerting and tools 
are less useful when conflicts occur suddenly 
and late. 

The global analysis showed that use of the 
Enhanced display in the head-on conflict, and 
in the two strategic lateral encounters, also led 
to increased minimum separations. 

Discounting the two altitude encounters (which 
did not require a resolution), these conflicts 
were present at the greatest distances from 
Ownship. This importance of range was 
furthered supported by the marginally 


significant effect of Range on minimum 
separation in the target analysis. 

Finally, it is interesting to note that allowing 
more time to detect a conflict resulted in more 
proactive responding for crews using either the 
Basic or the Enhanced display. Thus, crews 
responded earlier with and without the display 
enhancements when allowed to view the 
conflict developing over a longer period of 
time. However, care must be taken in 
interpreting this result. As noted earlier, every 
scenario contained a conflict and the crews 
were free to focus almost all of their attention 
on finding it. 

Context Traffic Data 

Minimum Separation: A global analysis of 
minimum separation from the Context (non- 
intruder) Traffic was conducted to determine 
if the Display Mode manipulation led to any 
differences in minimum separation with the 
Context Traffic. Initial examination of these 
data showed two extreme outliers which 
strongly skewed the distributions and means. 
Although an ANOVA yielded equivalent 
statistical outcomes with and without these 
outliers, they were removed from the data to 
obtain a better estimate of the mean 
performance. Figure 12 shows the average 
minimum separation between Ownship and 
Context Traffic as a function of Display Mode 
and Scenario. The main effect of Scenario 
was statistically significant (F(7,56) = 13.265, 
p < 0.001). This effect was probably due to 
the idiosyncratic nature of the various 
scenarios, since no particular care was taken to 
equalize separation from Context Traffic 
across scenarios with respect to initial 
trajectories, nor with respect to some nominal 
trajectory resolutions. There were no 
statistically significant effects of Display Mode 
or Display Mode by Scenario interaction (F < 
1.00) on minimum separation from Context 
Traffic. 
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Figure 13. Mean frequency of Ownship heading maneuvers versus Display Mode and Scenario. 


Discussion : There is no evidence that the 
display led to a change in minimum separation 
from Context Traffic. 

Maneuver Data 

Heading and altitude maneuvers were 
examined using the global ANOVA and the 
targeted ANOVA. Speed maneuvers were not 
examined since there were almost no changes 
to speed, averaging less than 0.2 per scenario. 
This was probably due to the long time lag 
involved in changing speed, and the relatively 
small amount by which a transport aircraft’s 
speed can be changed in the enroute 
environment. 

Heading Maneuver Analysis: There were, on 
average, 2.8 heading changes per scenario. 
Figure 14 shows the mean number of heading 
changes as a function of Scenario and Display 
Mode. The global analysis reveals a 
statistically significant main effect for Scenario 
(F(l,8)=2.221, p=0.046) and Scenario by 


Display Mode interaction (F(7,56)=2.436, 
p=0.03). In particular. Figure 14 shows that 
the Enhanced display led to a reduced number 
of heading maneuvers in the strategic and 
tactical right lateral scenarios and in the 
overtake scenario. The lack of any effect of 
Display Mode in the left lateral scenarios, and 
in the altitude change scenarios, is not 
surprising. The rules of the road specify that 
Ownship is not required to maneuver in the 
left lateral scenarios. Similarly, in the Altitude 
Lateral scenarios, when the Ownship crews 
contacted the crews of the Intruder aircraft, the 
Intruders agreed to level off 2000 feet above 
or below the Ownship, and again there was less 
need to maneuver. In these scenarios the 
Enhanced Display may show little or no 
impact, because there was little need for the 
Ownship to maneuver. 

While the global analysis of heading 
maneuvers suggests roles for both laterality 
(whether the Intruder approaches from the left 
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Figure 14. Mean frequency of Ownship altitude maneuvers versus Display Mode and Scenario. 


or right) and Display Mode, the targeted 
analysis did not generate any statistically 
significant effects. 

Altitude Maneuver Analysis : There were an 
average of 3.8 changes to altitude rate per 
scenario. While this appears to indicate that 
altitude maneuvers were quite frequent, closer 
examination of these data revealed that the 
majority of changes to altitude rate were due 
to two crews, one in each of the two Display 
Mode conditions. Both of these crews 
averaged about 12.5 altitude changes per 
scenario. During debriefing, it became 
apparent that they almost always changed 
altitude to resolve conflicts, because they 
determined that they had been placed well 
below the cruising altitude that offered 
optimum fuel efficiency. Thus, they always 
climbed to avoid any conflicts. When these 
crews were eliminated from the analysis the 
mean number of altitude change inputs 
dropped from 3.8 to 1.6 per scenario. Figure 
14 shows the mean number of altitude rate 
changes as a function of Scenario and Display 
Mode for the remaining four crews. 

The global analysis of these data reflect a 
statistically significant main effect for Scenario 
(F(l,8)=2.569, p=0.027), but no significant 
effects involving Display Mode. No 
statistically significant effects were found in 
the targeted analysis. While it appears that the 
effects of Scenario are due to an increased 
likelihood of making an altitude maneuver 
during horizontal lateral encounters, the 
reasons for this effect are not obvious. 


Discussion 

Display Mode appears to have only had an 
effect on the use of heading maneuvers, with 
crews apparently using fewer heading 
maneuvers when they had an Enhanced 
display. 

Crew Debriefing 

All crews were debriefed following the 
completion of the eight scenarios. This 
allowed for a spontaneous exchange of 
information. 

The debriefing provided much valuable, but 
not easily quantifiable, information about the 
strengths and weaknesses of the display. In 
general, the crews in both the Basic and the 
Enhanced Display conditions found their 
displays adequate to do their tasks, and they 
were generally enthusiastic (with 
qualifications) about using these displays for 
Free Flight. The following five categories 
reflect many of the comments that the authors 
feel provided the most valuable guidance for 
the present, and future, display designs. 

Clutter and Filtering: Clutter, and its 
associated impacts on detection and salience, 
emerged as an important issue in the 
debriefings. In particular, both data blocks 
and the predictors were noted as potential 
problems. Several pilots noted that the altitude 
and call sign text in the data blocks could be a 
problem. Some pilots liked flying with data 
blocks fully enabled (i.e., with call signs), but 
several pilots wanted call signs to be 
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individually selectable. Similarly, several 
pilots noted that having all predictors on at 
once led to a clutter problem, and saw 
individual selectability as a solution to the 
problem. 

Several pilots suggested dealing with clutter by 
filtering which aircraft were displayed on the 
basis of relevance. For example, it was 
suggested that altitude filtering could be 
implemented on the basis of proximity to 
Ownship altitude (e.g., allowing a pilot to 
select only aircraft within 4000 ft of Ownship). 
Another suggestion was to allow the crews to 
select just the aircraft below or above Ownship. 
Other suggestions were to filter out all aircraft 
except for those that were laterally converging 
on Ownship path, and/or those that were 
vertically converging on Ownship altitude. 
However, one crew also noted that people 
might forget that such selectable filters were 
enabled, thereby believing they were seeing 
the whole picture. This crew suggested that 
any such selectable filter only be enabled for a 
specified period of time, and then 
automatically revert to an unfiltered status. 
Finally, two crews suggested that aircraft based 
filtering could be accomplished without fully 
eliminating aircraft from a display. They 
suggested that it would be possible to 
differentially emphasize information on the 
basis of its probable relevance, such as by 
linking brightness to relevance. 

Tools : Since the tools were only made 
available to the crews using the Enhanced 
Display, almost all relevant comments came 
from these crews. Several of these crews said 
that they would like enhancements to the RAT 
that allow the evaluation of relative efficiency 
of maneuvers. Most crews noted that they 
made great use of the predictor tool, and two 
crews indicated they wanted it to remain on 
(not be cleared) after an SA 3 alert. Two crews 
also liked to leave the predictor tool on and set 
at ‘0’ so that a five mile diameter ring was 
always around all aircraft. 


Two crews noted problems with the RAT. One 
crew said that they tried to use the RAT 
initially, but after it twice required excessive 
turns they stopped using it. A second crew did 
not like that a conflict would sometimes 
reappear after a safe heading was found and 
the heading change was made using the 
heading selector on the mode control panel. 
The problem probably arose when they first 
made a turn that the RAT indicated would 
result in a clear path. This crew (and maybe 
others) did not fully understand that when 
they turned the RAT off, the system would 
once again begin evaluating the present course 
of the Ownship based on its velocity vector. If 
the velocity vector of the aircraft was not yet 
turned to a heading sufficiently clear of the 
conflict, the SA level symbology would return 
and remain on until the turn had aimed the 
velocity vector sufficiently clear of the 
conflict. These pilots’ comments suggest that 
it would be better to have alerts and SA levels 
that take into account the projected route of 
the Ownship (based on initiations of 
predetermined maneuvers) rather than make 
SA levels simply dependent upon the 
Ownship’ s instantaneous velocity vector. 

The Value of Early Information : Many 
crews’ comments reflected the value of 
information at an extended range. Several 
pilots said that information on aircraft beyond 
120 NM would be useful. Specifically, they 
thought that this would aid in planning 
(especially for altitude maneuvers) and would 
reduce the size of needed maneuvers. One 
crew even thought that a 640 NM range would 
be useful in oceanic flight. Only one crew (in 
the Basic Display condition) indicated that 
there was no payoff for maneuvering early, 
and this crew stated that they did not feel that 
it was their task to avoid an alert zone contact. 
Another crew in the Basic Display condition 
indirectly highlighted the importance of early 
information when they stated that the CTA 
(estimated time to closest approach), which did 
not appear until the SA 3 alert, determined their 
choice of maneuvers. 
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Information About Intent: Adding more 
specific intent information to the display was 
another concern of several pilots. In addition 
to the basic velocity vector information that 
was being used to project intent on the 
Enhanced display, the crews obtained intent 
information using radio communications. For 
example, critical intent information was only 
obtainable over the radio in the two scenarios 
where the intruding aircraft was descending or 
ascending to altitudes 2000 ft above or below 
Ownship (and thus posed no real threat). In 
other scenarios negotiations with the Intruder, 
or with other Context Traffic that were simply 
going to come close to Ownship, included 
specifications or agreements concerning intent 
(i.e., “I know you are there and will maintain 
my altitude and heading”). In light of this, 
one crew noted that display of digitally (non- 
voice) exchanged intent information is 
desirable when flying over non-English 
speaking countries. Among the enhancements 
to displayed intent information suggested by 
one or more crews were: 1) intended final 
altitude when an aircraft was changing altitude; 
2) intended course change (prior to course 
change); and 3) a trend vector on other 
aircraft in parallel flight and near minimum 
separation. Finally, one crew noted that, for 
the automated exchanges, it would be 
important to know who was sending and 
receiving intent information. 

Miscellaneous : For both display conditions 
there was a mixture of preferences expressed 
for relative or absolute altitude, with the often 
stated preference for selectable modes. Some 
crews also thought that turn rate and turn 
direction would be valuable additions to the 
display. Several crews in the basic display 
condition also expressed the desire for 
enhancements that would provide more 
temporally advanced conflict alerting, and/or 
predictive tools, with these enhancements 
being generally similar (and sometimes 
identical) to those provided with the Enhanced 
Display. 


General Discussion 

The results of the simulation have tended to 
validate the approach used in the design of the 
prototype CSD. Specifically, relative to the 
Basic display, the Enhanced display allowed 
more proactive responding by the crews. The 
Enhanced display was especially effective at 
this in scenarios where the crew were able to 
see the conflict developing over a longer 
interval. Furthermore, in scenarios where the 
initial distance between Ownship and Intruder 
was large, crews with the Enhanced display 
generated larger minimum separations. These 
results may be accounted for by the simple 
hypothesis that advanced information and 
good resolution tools make early resolutions 
possible. 

Furthermore, the data show no inherent 
tendency on the part of the crews to maneuver 
more often when given these enhancements, 
but instead shows an opposite trend for 
heading maneuvers. While there are other 
possibilities, this may suggest that the RAT 
allowed crews to more effectively plan lateral 
route adjustments. In contrast, the lack of any 
effect of Display Mode on altitude adjustments 
suggests that the RAT may have been a 
differentially effective planning tool across 
types of maneuvers. This would not be 
surprising since 1) the RAT did not provide an 
easy to use pictorial depiction of the proposed 
altitude maneuver (e.g. a profile view), and 2) 
the RAT evaluated the a change to altitude 
rate, but did not include leveling off at a 
specific altitude that is needed for a complete 
specification of the maneuver. Further 
research and development on the RAT is 
needed to evaluate and/or improve the utility 
of the display for altitude maneuver planning. 

However, the evaluation of this display has 
been done under very idealistic assumptions 
(already enumerated) that will rarely hold in a 
real enroute environment. For example, in this 
study the crews focused their attention almost 
entirely upon the CSD, and it provided nearly 
perfect conflict alerting. The value of the 
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alerting symbology must be assessed under 
conditions where there is incomplete and/or 
noisy surveillance information, and where 
conflicts are more rare. Also, the display must 
be evaluated in situations where the crew has 
other duties that take attention away from the 
CSD. 

The value of the individual design elements 
must also be assessed. The present CSD 
design incorporated a number of features 
(e.g., general traffic depiction, alerting 
symbology, RAT, and predictor tool). The 
relative importance of these individual features 
needs to be examined. 

The debriefings of the crews also provided 
some very good information that should be 
useful in directing future research and 
development on new display features. For 
example, clutter remains a problem and future 
work must address this. The present simulation 
was limited to a maximum of 10 aircraft on 
the display at one time, distributed across 9 
adjacent flight levels, and with no extraneous 
speed and heading changes. Some informal 
examinations, by the present authors, of the air 
traffic density in the Denver Center showed 
that it could rise to 20, and sometimes 30 
aircraft within 3 adjacent flight levels. 
Therefore some method of filtering or 
prioritizing what aircraft or symbology to 
display will be necessary. The data tags are a 
clearly a major source of clutter and methods 
for reducing this must be addressed. Totally 
removing them without providing some other 
form of altitude information is not an option. 
Methods of filtering out aircraft, or aircraft tag 
information, based on various measures of 
proximity or relevance was a primary solution 
suggested by the pilots. Another potential 
solution entails the use of alternative spatial 
formats (e.g., 3-D or multiple 2-D views) that 
more effectively/efficiently display altitude 
information. Another potentially effective 
method of dealing with clutter is to draw 
attention to important information by 
increasing the salience of symbols and text 
associated with that information, while 


decreasing the salience of the symbols 
associated with less important information 
(e.g., manipulation of brightness). 

Despite the clutter problem, the display of 
other types of information, primarily 
regarding intent, must also be considered and 
evaluated. Pilots suggested displaying 
information about how other aircraft are 
maneuvering, and providing advanced 
information indicating intent to maneuver. 
Furthermore, it is not clear if a 120 NM range 
is too little or too much (although no pilots 
indicated that it was too little). 

Finally, there was near unanimous agreement 
that predictors were desirable. However, there 
is still a need to present information that 
effectively shows both the predictions and the 
certainty of the predictions. The SA levels in 
the alerting system were designed to do this, 
but the extrapolations generated by the 
predictor tool were not. Furthermore, it is not 
clear in either case that the pilots were actually 
utilizing this information. A more controlled 
study is necessary to evaluate this. 

In conclusion, the comments and the 
performance data from this simulation are 
encouraging. They both show acceptance of 
the display and a moderate to substantial 
confidence among the pilots that they could, 
and would like to, do this task with this type of 
display. Further development and more 
controlled evaluations will be necessary, 
however, before the desired functionality of 
the CSD for any specific Free Flight 
implementation can be established. 
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Table 3. Descriptions of the conflicts in each of the eight scenarios. 


Table 3 

Descriptions of the conflicts in each of the eight scenarios. 

Scenario Name 

Scenario Description 

Right Strategic 
Lateral 

Intruder aircraft (ground speed = 473 knots) approaching on a collision 
course from the right at beginning of scenario, but too distant to 
immediately trigger any SA level. An SA, is triggered 8.2 minutes into the 
scenario, and this is followed by an SA 2 2.3 min later, and then by an SA 3 
2.2 minutes after that. In this scenario the Intruder has the right of way. 

Right Tactical 
Lateral 

Intruder aircraft (ground speed = 457 knots) to the right of Ownship and 
on a parallel course at beginning of scenario. Intruder turns onto collision 
course 9 minutes into the scenario. This triggers an SA, and then 
transitions through an SA 2 and an SA 3 (alert zone contact) within the next 
10 seconds. In this scenario the Intruder has the right of way. 

Left Strategic 
Lateral 

Intruder aircraft (ground speed = 473 knots) approaching on a collision 
course from the left at beginning of scenario, but too distant to immediately 
trigger any advanced SA level. An SA,is triggered 0.7 minutes into the 
scenario, and is followed by an SA 2 2.7 min later, and then by an SA 3 3.3 
min after that. In this scenario Ownship has the right of way. 

Left Tactical 
Lateral 

Intruder aircraft (ground speed = 473 knots) to the left of Ownship and on 
a parallel course at beginning of scenario. Intruder turns onto collision 
course 7.3 minutes into the scenario. This triggers an SA, and then 
transitions through and SA 2 and SA 3 (alert zone contact) within the next 15 
seconds. In this scenario Ownship has the right of way. 

Head-on 

Intruder aircraft (ground speed = 473 knots) on a head-on collision course 
at the beginning of the scenario, but is outside of surveillance range. The 
Intruder enters the surveillance range 8.2 minutes into the scenario at which 
point it immediately triggers an SA, which transitions to an SA 2 alert 20 
seconds after that. The Intruder finally triggers an S A 3 (alert zone contact) 
2.3 minutes following this. Right of way rules indicate that both the 
Intruder and the Ownship should maneuver to the right. 

Overtake 

Ownship overtaking Intruder aircraft (ground speed = 403 knots) from 
directly behind, and triggers an SA, 2.5 minutes into the scenario, which is 
followed by an SA 2 6 minutes later, and then an SA 3 (alert zone contact) 2.5 
minutes after the SA 2 . In this scenario Ownship has the right of way. 

Lateral Descent 

Intruder aircraft (ground speed = 445 knots) 4000 feet above and to the 
left of Ownship, and on a path that will pass directly above Ownship. 
Intruder begins descent toward Ownship 7.3 minutes into the scenario, 
triggering an immediate triggering an immediate SA,, which progresses 
through an SA 2 and then an SA 3 (alert zone contact) within about 8 
seconds. If contacted, the pseudo-pilot controlling the Intruder informs the 
Ownship crew of his intention to level off 2000 above below Ownship. This 
is accomplished 2.5 minutes later eliminating the SA 3 alert. 

Lateral Ascent 

Intruder aircraft (ground speed = 510 knots) is initially out of surveillance 
range. When Intruder appears, 2 minutes into the scenario, it is 4000 feet 
below and to the right of Ownship, and on a path that will pass directly 
below Ownship. Intruder begins ascent toward Ownship 2.5 minutes after it 
comes within surveillance range, triggering an immediate SA,, which 
progresses through an SA 2 and then an SA 3 (alert zone contact) within 
about 5 seconds. If contacted, the pseudo-pilot controlling the Intruder 
informs the Ownship crew of his intention to level off 2000 feet below 
Ownship. This is accomplished 2.25 minutes later eliminating the SA 3 alert. 
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